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Method for encoding and decoding video information, a motion compen- 
sated video encoder and a corresponding decoder 

The present invention relates to video coding. Li particular, it relates to com- 
5 pression of video information using motion compensated prediction. 

Background of the invention 

A video sequence consists of a large number video frames, which are formed 
10 of a large number of pixels each of which is represented by a set of digital 
bits. Because of the large number of pixels in a video frame and the large 
number of video frames even in a typical video sequence, the amount of data 
required to represent the video sequence quickly becomes large. For instance, 
a video frame may include an array of 640 by 480 pixels, each pixel having 
15 an RGB (red, green, blue) color representation of eight bits per color compo- 
nent, totaling 7,372,800 bits per frame. Video sequences comprise a sequence 
of still images, which are recorded/displayed at a rate of typically 15 - 30 
frames per second. The amount of data needed to transmit information about 
each pixel in each frame separately would thus be enormous. 

20 

Video coding tackles the problem of reducing the amount of information that 
needs to be transmitted in order to present the video sequence with an accept- 
able image quality. For example, in videotelephony the encoded video infor- 
mation is transmitted using conventional telephone networks, where transmis- 
25 sion bit rates are typically multiples of 64 kilobits/s. In mobile video- 
telephony, where transmission takes place at lesast in part over a radio com- 
munications link, the available transmission bit rates can be as low as 20 ki- 
lobits/s, 

30 In typical video sequences the change of the content of successive frames is 
to a great extent the result of the motion in the scene. This motion may be due 
to camera motion or due to motion of the objects present in the scene. There- 
fore typical video sequences are characterized by significant temporal correla- 
tion, which is highest along the trajectory of the motion. Efficient compres- 

35 sion of video sequences usually takes advantage of this property of video se- 
quences. Motion compensated prediction is a widely recognized technique for 
compression of video. It utilizes the fact that in a typical video sequence, im- 
age intensity/chrominance value in a particular frame segment can be pre- 
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dieted using image intensity/chrominance values of some other already coded 
and transmitted frame, given the motion trajectory between these two frames. 
Occasionally it is advisable to transmit a whole frame, to prevent the deterio- 
ration of image quality due to accumulation of errors and to provide addi- 
5 tional functionalities, for example, random access to the video sequence). 

A schematic diagram of an example video coding system using motion com- 
pensated prediction is shown in Figures 1 and 2 of the accompanying draw- 
ings. Figure 1 illustrates an encoder 10 employing motion compensation and 
10 Figure 2 illustrates a corresponding decoder 20. The operating principle of 
video coders using motion compensation is to minimize the prediction error 
frame E^x.y), which is the difference between the current frame I n (x 9 y) be- 
ing coded and a prediction frame P n (x 7 y) . The prediction error frame is thus 

15 E„(x,y) = I n {x,y)-P„(x y y). (1) 

The prediction frame is built using pixel values of a reference frame R^ix^y) , 
which is one of the previously coded and transmitted frames (for example, a 
frame preceding the current frame), and the motion of pixels between the cur- 

20 rent frame and the reference frame. The motion of the pixels may be pre- 
sented as the values of horizontal and vertical displacements Ax(x,y) and 
&y(x>y) of a pixel at location (x,y) in the current frame I„(x,y). The pair of 
numbers [&x(x y y\ Ay(x,y)] is called the motion vector of this pixel. The mo- 
tion vectors are typically represented using some known functions (called ba- 

25 sis functions) and coefficients (this is discussed in more detail below), and an 
approximate motion vector field (Ar(x,jy),Sy(x, i y)) can be constructed using 
the coefficients and the basis functions. 

The prediction frame is given by 

30 

P n (*,y) = + Mx,y\y + AKx, j)} C 2 ) 

where the reference frame <^(x,>0is available in the Frame Memory 17 of the 
encoder 10 and in the Frame memory 24 of the decoder 20 at a given instant. 
35 An information stream 2 carrying information about the motion vectors is 
combined with information about the prediction error (1) in the multiplexer 
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16 and an information stream (3) containing typically at least those two types 
of infomation is sent to the decoder 20. 

In the Prediction Error Coding block 14, the prediction error frame E n (x 7 y) is 
5 typically compressed by representing it as a finite series (transform) of some 
2-dimensional functions. For example, a 2-dimensional Discrete Cosine 
Transform (DCT) can be used. The transform coefficients related to each 
function are quantized and entropy coded before they are transmitted to the 
decoder (information stream 1 in Figure 1). Because of the error introduced 
10 by quantization, this operation usually produces some degradation in the pre- 
diction error frame E n (x,y) . 

In the Frame Memory 24 of the decoder 20 there is a previously reconstructed 
reference frame -R* (*,>>). Using the decoded motion information 

15 (Ax(x,y),Ay(x,y)) and R^fay)^ is possible to reconstruct the prediction 
frame P n (x,y) in the Motion Compensated Prediction block 21 of the decoder 
20. The transmitted transform coefficients of the prediction error frame 
E n ( x >y) are used in the Prediction Error Decoding block 22 to construct die 
decoded prediction error frame £„(*,.>>). The pixels of the decoded current 

20 frame I n ( x >y) are reconstructed by adding the prediction frame P m (x,y) and 
the decoded prediction error frame E n (x,y) 

t (x,y) = P« (*,y) + E n (x,y) = R n [x + lx(x,y\y + Ay(x 9 y)] + E n {x 7 y\ (3) 

25 This decoded current frame may be stored in the Frame Memory 24 as the 
next reference frame R n ^ (x,y) . 

Let us next discuss in more detail the motion compensation and transmission 
of motion information. The construction of the prediction frame P H (x,y)m the 

30 Motion Compensated Prediction block 13 of the encoder 10 requires informa- 
tion about the motion in the current frame I n (x,y). Motion vectors 
[&x(x,y), AX*,>0]are calculated in the Motion Field Estimation block 11 in 
the encoder 10. The set of motion vectors of all pixels of the current frame 
[AxQ, Ay(0] is called the motion vector field. Due to the very large number of 

35 pixels in a frame it is not efficient to transmit a. separate motion vector for 
each pixel to the decoder. Instead, in most video coding schemes the current 
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frame is divided into larger image segments and information about the seg- 
ments is transmitted to the decoder. 

The motion vector field is coded in the Morion Field Coding block 12 of the 
5 encoder 10. Motion Field Coding refers to representing the motion in a frame 
using some predetermined functions or, in other words, representing it with a 
model. Almost all of the motion vector field models commonly used are addi- 
tive motion models. Motion compensated video coding schemes may define 
the motion vectors of image segments by the following general formula: 

10 

1=0 

15 where coefficients a- and are called motion coefficients. They are 
transmitted to the decoder. Functions / and g % are called motion field basis 
functions, and they are known both to the encoder and decoder. 

In order to minimize the amount of information needed in sending the motion 
20 coefficients to the decoder, coefficients can be predicted from the coefficients 
of the neighboring segments. When this kind of motion field prediction is 
used, the motion field is expressed as a sum of a prediction motion field and 
refinement motion field. The prediction motion field uses the motion vectors 
associated with neighboring segments of the current frame. The prediction is 
25 performed using the same set of rules and possibly some auxiliary informa- 
tion in both encoder and decoder. The refinement motion field is coded, and 
the motion coefficients related to this refinement motion field are transmitted 
to the decoder. This approach typically results in savings in transmission bit 
rate. The dashed lines in Figure 1 illustrate some examples of the possible in- 
30 formation some motion estimation and coding schemes may require in the 
Motion Field Estimation block 1 1 and in the Motion Field Coding block 12. 

Polynomial motion models are a widely used family of models. (See, for ex- 
ample H. Nguyen and E, Dubois, "Representation of motion information for 
35 image coding/' in Proc. Picture Coding Symposium '90, Cambridge, Massa- 
chusetts, March 26-18, 1990, pp. 841-845 and Centre de Morphologie 
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Maihematique (CMM), "Segmentation algorithm by multicriteria region 
merging," Document SrM(95)19, COST 21 Iter Project Meeting, May 1995). 
The values of motion vectors are described by functions which are linear 
combinations of two dimensional polynomial functions. The translation^ mo- 
tion model is the simplest model and requires only two coefficients to de- 
scribe the motion vectors of each segment. The values of motion vectors are 
given by the formulae: 



4>>(x,j>) = *o 



(6) 



This model is widely used in various international standards (ISO MPEG-1, 
MPEG-2, MPEG-4, ITU-T Recommendations H.261 and H.263) to describe 
motion of 16x16 and 8x8 pixel blocks. Systems utilizing a translational mo- 
tion model typically perform motion estimation at full pixel resolution or 
15 some integer fraction of full pixel resolution, for example with an accuracy of 
1/2 or 1/3 pixel resolution. 

Two other widely used models are the affine motion model given by the 
equation: 

20 

Ax(x,y) = o 0 +a l x + a z y 

and the quadratic motion model given by the equation: 
25 (x,y) = a, + a } x + + a^xy + a A x 7 + ay 2 ^ 



The affine motion model presents a very convenient trade-off between the 
number of motion coefficients and prediction performance. It is capable of 
representing some of the common real-life motion types such as translation, 

30 rotation, zoom and shear with only a few coefficients. The quadratic motion 
model provides good prediction performance, but it is less popular in coding 
than the affine model, since it uses more motion coefficients, while the pre- 
diction performance is not substantially better. Furthermore, it is computa- 
tionally more costly to estimate the quadratic motion than to estimate the af- 

35 fine motion. 



When the motion field is estimated using higher order motion models (such as 
presented, for example, in equations 7 and 8), the motion field estimation re- 
sults in a motion field represented by real numbers. In this case the motion 
5 coefficients need to be quantized to a discrete accuracy before they are 
transmitted to the decoder. 

The Motion Field Estimation block 11 calculates motion vectors 
[Ar(x,>>), Ay(*,jO] of the pixels of a given segment S k which minimize some 

10 measure of prediction error in the segment. In the simplest case the motion 
field estimation uses the current frame I n {x,y) and the reference 
frame ^(r,^) as input values. Typically the Motion Field Estimation block 
outputs the motion field [&x(x y y\ Ay[x 7 y)\ to the Motion Field Coding block 
12. The Motion Field Coding block makes the final decisions on what kind of 

15 motion vector field is transmined to the decoder and how the motion vector 
field is coded. It can modify the motion model and motion coefficients in or- 
der to minimize the amount of information needed to describe a satisfactory 
motion vector field. 

20 The image quality of transmitted video frames depends on the accuracy with 
which the prediction frame can be constructed, in other words on the accu- 
racy of the transmitted motion information, and on the accuracy with which 
the prediction error information is transmitted. Here the term accuracy refers 
not only to the ability of the emotion field model to represent the motion 

25 within the frame but also to the numrical precision with which the motion in- 
formation and the prediciton error information is represented. Motion infor- 
mation transmitted with hich accuracy may be canceled out in the decoding 
phase due to low accuracy of the precidiction error frame, or vice versa. 

30 Current video coding systems employ various motion estimation and coding 
techniques, as discussed above. The accuracy of the motion information and 
the transmission bit rate needed to transmit the motion information are typi- 
cally dictated by the choice of the motion estimation and coding technique, 
and a chosen technique is usually applied to a whole video sequence. Gener- 

35 ally, as the accuracy of the transmitted motion information increases, the 
amount of transmitted information increases. 
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In general, better image quality requires larger amounts of transmitted infor- 
mation. Typically, if the available transmission bit rate is limited, this limita- 
tion dictates the best possible image quality of transmitted video frames. It is 
also possible to aim for a certain target image quality, and the transmission bit 
rate then depends on the target image quality. In current video coding and de- 
coding systems, the trade-offs between the required transmission bit rate and 
image quality are mainly made by adjusting the accuracy of the information 
presenting the prediction error frame. This accuracy may change, for exam- 
ple, from frame to frame, or even between different segments of a frame. 



The problem in changing the accuracy of the transmitted prediction error 
frame is that it may cause unexpected degradation of the overall performance 
of the video encoding, for example, when conforming to a new available 
transmission bit rate. In other words, the image quality achieved is not as 
15 good as that expected considering the transmission bit rate. The image quality 
may deteriorate drastically, when a lower transmission bit rate is available, or 
the image quality may not be enhanced even though a higher transmission bit 
rate is used. 

20 SUMMARY OF THE INVENTION 

The object of the invention is to provide a flexible and versatile motion com- 
pensated method for encoding/decoding video information. A further object 
of the invention is to provide a method that ensures good transmitted video 
25 quality for various transmission bit rates. A further object is that the method 
may employ various motion estimation and coding techniques. 

These and other objects of the invention are achieved by selecting the quanti- 
zation accuracy of the motion coefficients so that the accuracy of the trans- 
30 mitted motion information is compatible with the accuracy of the prediction 
error information. 

A method according to the invention is a method for encoding video informa- 
tion, comprising the following steps of: 
35 - estimating the motion of picture elements between a piece of reference 
video information and a piece of current video information, 
- modeling the motion of picture elements using a certain set of basis func- 
tions and certain motion coefficients, 
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- defining a certain set of quantizers, 

- selecting, based on a certain predetermined selection criterion, a motion co- 
efficient quantizer from the sec of quantizers, and 

- quantizing the motion coefficients using the selected motion coefficient 
5 quantizer. 

In a method according to the invention, the motion of picture elements be- 
tween a certain piece of reference video information and a piece of current 
video information is estimated. The resulting motion vector field is repre- 

10 senred using certain basis functions and motion coefficients. The basis func- 
tions are known both to the encoder and decoder, so the transmission of said 
coefficients enables the decoder to determine an estimate for the motion of 
picture elements. Typically the coefficient values are real numbers, and there- 
fore quantization is needed in order to present the coefficients to a certain 

15 discrete accuracy using a certain number of bits. The coefficients are quan- 
tized before transmission to the decoder, or before using them in constructing 
a prediction frame. 

In a method according to the invention, a set of quantizers is provided. Here 
20 the term quantizer refers to a function mapping real numbers to certain recon- 
struction values. For each reconstruction value there is a quantization interval 
determining the range of real numbers which are mapped/quantized to said 
reconstruction value. The size of the quantization intervals can be, for exam- 
ple, the same for each reconstruction value (uniform quantizer) or the size of 
25 the quantization interval can be different for each reconstruction value (non- 
uniform quantizer). The quantization interval determines the accuracy with 
which the coefficients are represented. The quantizers in the set may all be 
similar so that the reconstruction values and the quantization intervals are 
scaled from quantizer to quantizer using a certain parameter. It is also possi- 
30 ble that the set of quantizers comprises different types of quantizer, both uni- 
form and non-uniform, for example. Quantizers are further discussed in the 
detailed description of the invention. 

The selection criterion for the quantizer in a method according to the inven- 
35.. . tion can be, for example, the target image quality or the target transmission 
bit rate. The selection of a quantizer may also be bound to some other vari- 
able, which depends on the target image quality or the transmission bit rate. 
A new quantizer can be selected, for example, each time the target transmis- 
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sion bit rate changes. For various parts of a video frame, for example, it is 
possible the use different quantizers. 

According to the invention, when the target image quality changes, it is pos- 
5 sible to adjust the accuracy with which both the prediction error information 
and the motion information is encoded and transmitted. Therefore, for each 
image quality or transmission bit rate, it is possible to obtain good overall 
coding performance. It is possible, for example, to adjust both the accuracy of 
the prediction error information and that of the motion information to be able 

10 to transmit the encoded video stream using a certain bit rate, Ir is also possi- 
ble, for example, that the accuracy of the prediction error information is dic- 
tated by the target image quality, and in the method according to the invention 
the accuracy of the motion information is adjusted to be compatible with the 
prediction error accuracy: quantization should not be too fine, because the 

15 motion information cannot enhance image quality beyond the accuracy pro- 
vided by the prediction error information, but not too coarse either, because 
coarsely quantized motion information may deteriorate the image quality pro- 
vided by the prediction error information. 

20 A method according to the invention does not pose any restrictions on the 
motion field estimation or motion field coding techniques used to obtain the 
motion coefficients. It is therefore applicable with any such techniques. For 
example, motion model adaptation may be used to avoid the cost of using an 
over accurate motion field estimation and coding technique by providing a se- 

25 lection of motion field estimation and/or coding techniques providing various 
accuracies. An appropriate motion field estimation and/or coding te chni que 
can then be selected based on the target image quality or target bit rate, and 
fine-tuning between the prediction error information accuracy and the motion 
information accuracy can be performed by selecting a proper quantizer. 

30 

The invention can be straightforwardly applied to existing motion compen- 
sated video coding methods and systems. In such prior-art systems, the quan- 
tization of motion coefficients is typically done to a certain, predetermined 
accuracy, which works well for a certain target image quality. When the tar- 
35 get image quality or the available transmission bit rate differs remarkably 
from the designed image quality, the video encoder produces worse image 
quality than that which could be achieved for a given transmission bit rate. 
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This effect can be eliminated by selecting a more appropriate quantizer for 
the motion coefficients according to the invention. 

The invention also relates to a method for decoding encoded video informa- 
5 tion, comprising the following steps of: 

- receiving quantized motion coefficients describing motion of picture ele- 
ments, 

- defining a set of inverse quantizers, 

- determining a selected motion coefficient quantizer using which the motion 
10 coefficients are quantized, 

- performing inverse quantization of the quantized motion coefficients using 
an inverse quantizer corresponding to the selected motion coefficient quan- 
tizer, 

- determining the motion of the picture elements using the inverse quantized 
15 motion coefficients and certain basis functions, and 

- determining a piece of prediction video information using a piece of refer- 
ence video information and the determined motion of the picture elements. 

An encoder according to the invention is an encoder for performing motion 
20 compensated encoding of video information, comprising: 

- means for receiving a piece of current video information, 

- memory means for storing a piece of reference video information, 

- motion field estimation means for estimating a motion field of picture ele- 
ments in the piece of current video information using at least the piece of ref- 

25 erence video information, 

- motion field coding means, which comprise 

• means for producing motion coefficients describing the estimated mo- 
tion field, 

- first selection means for selecting a quantizer from a set of quantizers, 
30 said first selection means having an input to receive information indicat- 
ing a selection criterion and an output to send information indicating the 
selected quantizer, and 

- quantization means for quantizing motion coefficients using the se- 
lected quantizer, said quantization means having an input to receive in- 

35 formation indicating the selected quantizer, a second input to receive the 

motion coefficients, and an output to send the quantized motion coeffi- 
cients, and 

- motion compensated prediction means, which means comprise 
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- second selection means for selecting an inverse quantizer from a set of 
inverse quantizers, said second selection means having an input to re- 
ceive information indicating a selection criterion and an output to send 
information indicating the selected inverse quantizer, 

5 - inverse quantization means for inversely quantizing the quantized mo- 

tion coefficients using the selected inverse quantizer, said quantization 
means having an input to receive the quantized motion coefficients, a 
second input to receive information indicating the selected inverse quan- 
tizer and an output to send the inverse quantized motion coefficient, and 
10 - means for determining a a piece of prediction video information using 

at least the piece of reference video information and the inverse quan- 
tized motion coefficients. 

The invention relates further to a decoder for performing the decoding of en- 
15 coded video information, comprising: 

- memory means for storing a piece of reference video information, 

- input means for receiving quantized motion coefficients, and 

- motion compensated prediction means, which comprise 

- selection means for selecting an inverse quantizer from a set of inverse 
20 quantizers, said second selection means having an input to receive in- 
formation indicating a selection criterion and an output to send informa- 
tion indicating the selected inverse quantizer, 

- inverse quantization means for inversely quantizing the quantized mo- 
tion coefficients using the selected inverse quantizer, said quantization 

25 means having an input to receive the quantized motion coefficients, a 

second input to receive information indicating the selected inverse quan- 
tizer and an output to send the inverse quantized motion coefficient, and 

- prediction means for determining a piece of prediction video informa- 
tion using at least the piece of reference video information and the in- 

30 verse quantized motion coefficients. 

In one advantageous embodiment of the invention, the encoder and decoder 
are combined to form a codec. The motion compensated prediction parts of 
the encoder and decoder are similar, and they may be provided by a common 
35 part, which is arranged to operate as a part of the encoder and a part of the 
decoder, for example, alternatingly. 
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The invention relates further to a computer program element for performing 
motion compensated encoding of video information, comprising: 

- means for receiving a piece of current video information, 

- memory means for storing a piece of reference video information, 

- motion field estimation means for estimating a motion field of picture ele- 
ments in the piece of current video information using at least the piece of ref- 
erence video information, 

- motion field coding means, which comprise 

- means for producing motion coefficients describing the estimated mo- 
tion field, 

- first selection means for selecting a quantizer from a set of quantizers, 
said first selection means having an input to receive information indicat- 
ing a selection criterion and an output to send information indicating the 
selected quantizer, and 

_ quantization means for quantizing motion coefficients using the se- 
lected quantizer, said quantization means having an input to receive in- 
formation indicating the selected quantizer, a second input to receive the 
motion coefficients, and an output to send the quantized motion coeffi- 
cients, and 

- motion compensated prediction means, which means comprise 

- second selection means for selecting an inverse quantizer from a set of 
inverse quantizers, said second selection means having an input to re- 
ceive information indicating a selection criterion and an output to send 
information indicating the selected inverse quantizer, 

- inverse quantization means for inversely quantizing the quantized mo- 
tion coefficients using the selected inverse quantizer, said quantization 
means having an input to receive the quantized motion coefficients, a 
second input to receive information indicating the selected inverse quan- 
tizer and an output to send the inverse quantized motion coefficient, and 

- means for dete rminin g a piece of prediction video information using at 
least the piece of reference video information and the inverse quantized 
motion coefficients. 

A second computer program element according to the invention is a computer 
program element for performing the decoding of encoded video information, 
comprising: 

- memory means for storing a piece of reference video information, 

- input means for receiving quantized motion coefficients, and 
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- motion compensated prediction means, which comprise 

- selection means for selecting an inverse quantizer from a set of inverse 
quantizers, said second selection means having an input to receive in- 
formation indicating a selection criterion and an output to send informa- 

5 tion indicating the selected inverse quantizer, 

• inverse quantization means for inversely quantizing the quantized mo- 
tion coefficients using the selected inverse quantizer, said quantization 
means having an input to receive the quantized motion coefficients, a 
second input to receive information indicating the selected inverse quan- 
10 tizer and an output to send the inverse quantized motion coefficient, and 

- prediction means for deter mining a piece of prediction video informa- 
tion using at least the piece of reference video information and the in- 
verse quantized motion coefficients. 

15 According to one advantageous embodiment of the invention, a computer 
program element as specified above is embodied on a computer readable me- 
dium. 

The novel features which are considered as characteristic of the invention are 
20 set forth in particular in the appended Claims. The invention itself, however, 
both as to its construction and its method of operation, together with addi- 
tional objects and advantages thereof, will be best understood from the fol- 
lowing description of specific embodiments when read in connection with the 
accompanying drawings. 

25 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates an encoder for motion compensated coding of video ac- 
cording to prior art, 

30 

Figure 2 illustrates a decoder for moticii compensated coding of video ac- 
cording to prior art, 

Figure 3 illustrates a set of uniform quantizers, 

35 . 

Figure 4 illustrates a modified uniform quantizer, 
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Figure 5 illustrates a flowchart of a motion compensated video encoding 
method according to the invention, 

Figure 6 illustrates a flowchart of a motion compensated video decoding 
method according to the invention, 

Figure 7 illustrates the motion field encoding and motion compensated pre- 
diction blocks of an encoder according to a first preferred em- 
bodiment of the invention, 

Figure 8 illustrates an encoder for motion compensated encoding of video 
according to a second preferred embodiment of the invention, 

Figure 9 illustrates a decoder for motion compensated decoding of video 
according to the second preferred embodiment of the invention, 
and 

Figure 10 illustrates a motion field encoding block of an encoder according 
to a third preferred embodiment of the invention. 



DETAILLED DESCRIPTION 

Figures 1 and 2 are discussed in detail in the description of motion compen- 
sated video encoding and decoding according to prior art. 

In encoding and decoding methods according to the invention, a set of quan- 
tizers is defined. Different quantizers may differ in the way reconstruction 
values and intervals are selected. Quantization of a motion coefficient a, can 
be performed e.g. by utilizing following equation: 



a. ~ 



' 2 



(9) 



where q is a quantization parameter defining the interval between reconstruc- 
tion points and |_*J is the 'floor 1 operation (largest integer not larger thanx). 
In this case, the quantization output a % is an integer index that can be entropy 
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coded and transmitted to a decoder. In this example, the corresponding in- 
verse quantization operation can be defined by 

a, = qa i (10) 

5 

and the inverse quantized motion coefficients a i are used to approximate the 
original coefficients a x . Equations 9 and 10 define a uniform quantizer with 
evenly distributed reconstruction values and reconstruction intervals of equal 
size. Figure 3 illustrates three uniform quantizers having different values of q: 
10 q u q 2 and $ 3 . 



Figure 4 illustrates a modified uniform quantizer. In this quantizer, parameter 
q defines the reconstruction points. The quantization interval, in which the 
coefficient a, values are mapped to <S, = 0, is larger than the other quantization 
15 intervals, which are of constant size. This quantizer can be defined, for exam- 
ple, using an additional parameter e, which is half of the increase in the quan- 
tization interval for 4 = 0. Quantization according to the modified uniform 
quantizer can be defined, for example, by 
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a. = 



a t -sgn(» + | 



where sgn(a^) gives the sign of Q t . The corresponding inverse quantization op- 
eration can be defined, for example, using equation 10. 



25 When the modified uniform quantizer is used, the transmission of quantized 
coefficients d, is likely to require fewer bits. This is because the quantization 
interval corresponding to values near zero is larger than in the uniform quan- 
tizer, and typically a zero-valued quantized coefficient can be transmitted us- 
ing a smaller number of bits than quantized coefficients having non-zero val- 

30 ues. 



Above,, two -different quantizers are described as examples of quantizers that 
can be used in a method according to the invention. In quantization, the size 
of the quantization interval is inversely proportional to the number of bits 
35 needed to represent the quantized coefficients d,. The quantization interval 



16 

also determines the accuracy according to which the inverse quantized coeffi- 
cients di correspond to the original coefficients a,. The quantization interval 
thus represents the trade-off between quantization accuracy and the amount of 
information needed in transmitting motion coefficients to the decoder. In a 
5 method according to the invention, a set of quantizers is defined for quantiz- 
ing the motion coefficients. The set may comprise, for example, various uni- 
form quantizers having different values of q. 

Figure 5 presents a flowchart of a motion compensated video encoding 
10 method according to the invention. In step 501, a set of quantizers for quan- 
tizing the motion coefficients is defined. The quantizers may be, for example, 
uniform quantizers having various quantization intervals, that is different val- 
ues of parameter q. When uniform quantizers are used, it is enough to define 
the allowed quantization intervals in step 501. In step 502 an appropriate cri- 
15 terion for selecting the quantizer is defined. This criterion can be, for exam- 
ple, the target image quality or the target bit rate. It is also possible to select 
the motion information q uant izer based on a previously selected prediction 
error quantizer. In step 503 the current frame I n {xy) is received. In step 504 
motion in the current frame is estimated and modeled, for example, segment 
20 by segment using certain basis functions ffay) and giixy), and coefficients a, 
and b t . The motion vector field is thus represented as 

am 

Ar(*.jO= 5>i/.(*.j0 

1=0 
A/-1 

ICO 

25 

Selecting the quantizer according to the invention does not pose any restric- 
tions on the motion field estimation and/or coding methods that can be used. 
The estimation and coding of the motion vector field can be made, for exam- 
ple, using any techniques used in prior art motion compensated video coding 

30 schemes. The basis functions used in coding the motion field vector are most 
preferably orthogonal, because orthogonal functions are less sensitive to in- 
accuracies in the decoded motion coefficients caused by quantization. Any 
basis function, however, may be used. Furthermore, the motion vector field 
that is coded can be, for example, a refinement motion vector field which is 

35 used to correct a prediction motion vector field constructed, for example, by 
using the motion of neighboring segments within the frame. 
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When a translational model is used to represent the motion vector field, the 
estimation of the motion vector field is typically carried out using a certain 
accuracy. This accuracy is the upper limit (that, is, the best possible accuracy) 
5 with which the motion coefficients a 0 and b 0 can be determined. Using quan- 
tization according to the invention, it is possible to lower the accuracy with 
which the motion coefficients are tra nsmi tted to a receiver. 

In step 505 the quantizer to be used in the quantization of the motion coeffi- 
10 cients a s and b> is selected and in step 506 the motion coefficient are quan- 
tized, resulting in the quantized coefficients a, and b. . They are transmitted 
to a receiver in step 51 1. Advantageously the receiver determines the motion 
information quantizer with reference to, for example, the quantizer (QP) used 
to encode the prediction error infonnation or the target bit rate of the encoded 
15 video stream. If the motion coefficient quantizer is chosen with reference to 
the quantizer QP used to encode die prediction error information, the receiver 
is able to deter min e the motion coefficient quantizer implicitly. In this case an 
explicit indication of the selected value of q, for example, is not required. It 
is, however, an alternative to transmit infonnation relateing to q in step 512. 
20 It can be transmitted as pan of the encoded video frame or it can be signaled 
independently to the receiver. Furthermore, the receiver may determine the 
selected quantizer, for example, from the format in which the quantized coef- 
ficients are transmitted. 

25 In step 507 a prediction frame PJxy) can be determined using a reference 
frame Rn(xy) 

P n (x 9 y) = R n (x + Zx(x t y) 9 y + Ay(x,y)) . 

30 Inverse quantized coefficients a, and b { are typically used in constructing 
Zx{x,y) = Z ff i/*( x »>) ^ 4K*>.y) = £5*i(*.j0- Thereafter the prediction er- 
ror frame E n (x,y) = I&yyP n (xy) can be determined (step 508), and it can be 
encoded (step 509). After encoding, the prediction error frame can be repre- 
sented using prediction error coefficients c h In step 510 the prediction error 

35 coefficients are quantized using a second quantizer. The resulting quantized 
prediction error coefficients c are transmitted to the receiver in step 511. The 
selection of the second quantizer is out of the scope of the present invention. 
Typically its selection is dictated by the target image quality. It may alterna- 
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tively be defined according to the requirements of a video coding standard, 
for example ITU-T Recommendation H.261 or H.263. 

Above, frames as used as an example of a piece of video information. In en- 
5 coding and decoding methods and apparatus according to the invention, it is 
also possible to transmit and process video information, for example, block- 
by-block or segment-by-segment in addition to the frame-by-frame approach. 

Figure 6 illustrates a flowchart of a decoding method for motion compensated 

10 coded video frames according to the invention. In step 601 a set of inverse 
quantizers is defined, and in step 602 the quantized coefficients a s and £, are 
received. In step 603 the motion information quantizer is determined. It can 
be determined, for example, by determining from the received prediction er- 
ror coding data the prediction error quantizer, and selecting an inverse quan- 

15 tizer for motion coefficients according to a predefined set of rules. Alterna- 
tively, the inverse quantizer may be selected according to an explicit indica- 
tion in the received bit stream. Thereafter inverse quantization of the motion 
coefficients is canied out (step 604), resulting in the inverse quantized motion 
coefficients d, and % . In step 605, the basis function ffccy) and gi(xy) are de- 

20 termined. The basis function may be determined a priori if the motion estima- 
tion and coding technique uses only certain basis functions. It is also possible 
that the motion estimation and coding technique uses motion model adapta- 
tion, in which case information about the selected basis functions is transmit- 
ted to the receiver.Information about these can be, for example, transmitted 

25 within the encoded frame, or the basis functions may be otherwise known to 
both the encoding and decoding method. 

In step 606 the motion of picture elements is determined using at least the in- 
verse quantized motion coefficients and the basis functions. Thereafter the 

30 prediction frame P n (xy) can be determined in step 607. In step 608 the quan- 
tized prediction error coefficients c, are dequantized using an inverse quan- 
tizer related to the prediction error. In step 609 a decoded prediction error 
frame E n (x,y) is constructed using the inverse quantized prediction error co- 
efficients c t . In step 610 the decoded current frame is determined, for exam- 

35 pie, by I n (x,y) = P n (x,y) +E n (x t y) . 

Figure 7 presents a schematic drawing of a motion field coding block and a 
motion compensated prediction block of a motion compensated video encoder 
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according to a first preferred embodiment of the invention. In the motion field 
coding block IT motion field coding is carried out using the motion vector 
field coding block 51. The output of the motion vector field coding block 
comprises the motion coefficients a, and b§. The quantization block 30 is re- 
5 sponsible for the quantization of the motion coefficients. It performs the 
quantization using a selected quantizer 31. The quantizer selection block 32 
comprises a set of quantizers 33, from which the currently used quantizer is 
selected. The quantizer selection criterion block 34 outputs the current crite- 
rion for the selection. It may, for example, indicate the current target image 
10 quality or the target bit rate, and the quantizer selection block 32 selects an 
appropriate quantizer based on the criterion. 

In the motion compensated prediction block 13 f according to the invention, 
there is an Inverse Quantization Selection block 42, comprising a set of in- 
15 verse quantizers 43. Inverse quantization is carried out in the Inverse Quanti- 
zation block 40, using the selected inverse quantizer 41. The inverse quan- 
tized motion coefficients d, and Z^are typically further used in the Motion 
Vector Field Decoding block 52 for determining a prediction frame P n (xy). 

20 Information about the quantized motion coefficients is transmitted from the 
Quantization block 30 to the Inverse Quantization block 40 in the Motion 
Compensated Prediction block 13'. In Figure 7, information about the se- 
lected quantizer is transmitted from the Quantizer Selection block 32 to the 
Inverse Quantizer Selection block 42 in the Motion Compensated Prediction 

25 block 13' in the encoder. 

A decoder according to the invention comprises a Motion Compensated Pre- 
diction block 2V which is similar to the Motion Compensated Prediction 
block 13' in an encoder according to the invention. The Inverse Quantization 
30 Selection block 42 in the decoder may receive separate information from 
which it can determine the correct inverse quantizer from the encoder or, al- 
ternatively, it may receive the information from some other part of the de- 
coder. 

35 Figure 8 presents a schematic diagram of a motion compensated video en- 
coder according to a second preferred embodiment of the invention. In the 
encoder 10' according to the invention there is Motion Field Coding block 12', 
which comprises a Quantization block 30 and a Quantizer Selection block 32. 
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The Motion Compensated Prediction block 13' of the encoder 10' comprises 
an Inverse Quantization block 40 and an Inverse Quantizer Selection block 
42. The quantized coefficients (arrow 2) and information about the prediction 
error (arrow 1) are transmitted to a decoder. 

5 , 

In an encoder according to the second preferred embodiment of the invention, 
the selection of the motion coefficient quantizer is dictated by the quantizer 
QP used in the Prediction Error Coding block 14 of the encoder. The predic- 
tion error quantizer QP can be selected, for example, based on the target im- 

10 age quality. By binding the selection of the quantizer used in quantizing the 
motion coefficients to the quantizer used in the coding of the prediction error, 
the target image quality of the encoded video frames can be reliably attained. 
The Prediction error Coding block 14 may indicate the current prediction er- 
ror quantizer (QP) to the Quantizer Selection block 32 and also to the Inverse 

15 Quantizer selection block 42 in the encoder 10', The Quantizer Selection 
block 32 may alternatively inform the Inverse Quantization Selection block 
42 in the encoder 10' about the prediction error quantizer. 

The Quantizer Selection block 32 and the Inverse Quantizer Selection block 
20 42 may alternatively reside, for example, in the Prediction Error Coding block 
14 of the encoder. 

In a decoder accoding to the second preferred emdobiment of the invention, 
the decoder receives information about the quantizer used in coding the pre- 

25 diction error frame and thus there usually is no need to transmit explicit in- 
formation concerning the quantizer of the motion coefficients. The decoder 
may determine the motion coefficient quantizer using the information trans- 
mined from the Prediction Error Coding block 14 (arrow 1) or from corre- 
sponding signaling information. The information streams 1 and 2 are typically 

30 multiplexed to an information stream 3 representing the encoded video frame. 
This information stream is transmitted to a decoder according to the inven- 
tion. The information about the prediction error quantizer may alternatively 
reside in the decoder and require no communication between the encoder and 
decoder. 

35 

The dashed arrows pointing to the Motion Field Coding blocks in Figure 8 
present some possible examples of information streams within the encoder. 
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The details of the structure of the encoder depend on the particular motion 
field estimation and coding techniques employed. 

Figure 9 presents a schematic diagram of a decoder 20' for decoding motion 
5 compensation coded video frames according to the second preferred embodi- 
ment of the invention. It comprises a Motion Compensated Prediction block 
2l\ where there is an Inverse quantization block 40 according to the invention 
and an Inverse Quantizer Selection block 42 according to the invention. Typi- 
cally the Prediction Error Decoding block 22' informs the Inverse Quantizer 

10 Selection block 42 about the selected quantizer QP used to quantize the pre- 
diction error information. This Inverse Quantization Selection block 42 then 
selects an appropriate inverse quantizer for the motion coefficient information 
on the basis of the QP value it receives from the Prediction Error Decoding 
block. The Inverse Quantizer Selection block 42 may alternatively reside, for 

15 example, within the Prediction Error Decoding block 22 of the decoder. 

Figure 10 presents a more detailed schematic diagram of a Motion Field Cod- 
ing block 12* according to a third preferred embodiment of the invention. 
The main idea in this motion field coding block is to present the motion vec- 

20 tor field as a sum of two vector fields: a vector field that can be predicted 
from other segments in the frame ^(xj) and Ay pfd (x,y)md a refinement 
motion vector field Ax^^ (x,y) and Ay^^ (x,y) . We refer to the patent 
application US 09/371641 concerning details of the motion vector field esti- 
mation and coding in a Motion Field Coding block 12* according to the third 

25 preferred embodiment of the invention. Here we only briefly describe the 
coding of the motion vector field in block 12* at a general level. 

In block 12* there is a Motion Analyser block 61 and a block 62, which is re- 
sponsible for the selection of a neighboring segment and for building the Pre- 

30 diction Motion Field. Typically, when predicting the motion of a certain seg- 
ment, the motion of at least one of its neighboring segments is used. This pre- 
diction is then refined by deter minin g a refinement motion vector field, which 
is coded, for example, using an affine model employing orthogonal basis 
functions. The Motion Coefficient Removal block 63 basically determines, 

35 which refinement motion coefficients can be set to zero without causing the 
accuracy of the morion vector field to deteriorate to an unacceptable level. 
The refinement motion coefficients, remaining after the removal procedure, 
are quantized in a Quantization block 30 according to the invention. 
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The selection of the quantizer used to quantize motion coefficients can be 
dictated by the choice of the quantizer used in the Prediction Error Coding 
block of the encoder. The Prediction Error Coding can be performed accord- 
5 ing to the ITU-T Recommendation H.263 using 2-dimensional Discrete Co- 
sine Transform and defining two uniform quantizer QP for the frame: QP P 
stays constant over the whole frame and QP S may vary for each 16 x 16 pixel 
segment in the frame. 

10 In the third preferred embodiment of the in invention, the set of quantizers 
consists of unif orm quantizers having different quantization intervals. There- 
fore, the selected motion coefficient quantizer can be specified by its quanti- 
zation interval. Thie quantization interval, in turn, is defined by parameter q. 
This parameter can be dependent, for example, on the quantization parameter 

15 QPp. In Table 1, q denotes the quantization interval in the motion coefficient 
quantization and QP P denotes half of the quantization interval of the predic- 
tion error coefficients. Motion coefficients in block 12* can be quantized ac- 
cording to Table 1, in other words, by selecting the quantizer in block 30 
based on the value of QP P . Thereafter the quantized motion coefficients can 

20 be entropy coded and transmitted to a decoder. 
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Table 1. Relationship between the quantization interval q related to motion 
coefficients and quantization interval QPp related to prediction error coeffi- 
cients. 
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As can be seen from Table 1, the size of the quantization interval used to 
quantize the motion coefficients is related to the quantization interval used in 
the quantization of the prediction error coefficients. This means that when the 
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prediction error is transmitted using a higher accuracy, the motion informa- 
tion is automatically transmitted using a h ; gher accuracy and vice versa. The 
value of QPp can be selected, for example, based on the target image quality. 
Once the value of QPp is known, it is possible straightforwardly to determine 
5 a quantizer and an inverse quantizer corresponding tq said QP P value. 

It is also possible to have a set of quantizers which comprises uniform and 
non-uniform quantizers. In this case, it is also possible to select a quantizer 
depending on the value of QPp. 

10 

An encoder or a decoder according to the invention can be realised using 
hardware or software, or using a suitable combination of both. An encoder or 
decoder implemented in software may be, for example, a separate program or 
a software building block that can be used by various programs. The Quanti- 
15 zation block, Quantizer Selection block and Quantizer Selection Criterion 
block are in the above description and in the drawings represented as separate 
units, but the functionality of these blocks can be implemented, for example, 
in one software program unit responsible for the selection of a quantizer and 
quantization of the motion coefficients. 

20 

It is also possible to implement an encoder according to the invention and a 
decoder according to the invention in one functional unit. Such a unit is called 
a codec. A codec according to the invention may be a computer program or a 
computer program element, or it may impletemented at least partly using 
25 hardware. 

In view of the foregoing description it will be evident to a person skilled in 
the art that various modifications may be made within the scope of the inven- 
tion. While a number of preferred embodiments of the invention have been 
30 described in detail, it should be apparent that many modifications and varia- 
tions thereto are possible, all of which fall within the true spirit and scope of 
the invention. 



